The present disclosure is related to enhancing user interaction with software applications running on a electronic device. More specifically, the present disclosure is related to provide haptic feedback to enhance user interaction with a software application running on a portable electronic device.
To provide haptic feedback, software applications typically implement underlying haptic techniques with whatever associated hardware is available. For example, an eccentric rotating mass (ERM) actuator may be activated to provide a vibration in a electronic device, or in a peripheral device associated with an electronic device, e.g., a joystick or gamepad for a video game console. To implement such a haptic technique, the software typically requires low level interfacing with the hardware in order to drive it at a fundamental level. Additionally, the software typically requires actual control over the hardware in order to generate tactile and kinesthetic feedback, so that tactile and kinesthetic haptic feedback and effects can be generated.
Additionally, to provide haptic feedback, the software for providing haptic feedback should interact with and provide an application the ability to use the haptic feedback capabilities provided by the hardware and the haptic software. Typically, application programming interfaces (APIs) or software development kits (SDKs) are developed and used by programmers to utilize the hardware and haptic software. These APIs and SDKs typically require application developers to modify their program's source code in order to use the haptic functionality. Given this type of integration into an application's source code, specific haptic effects can be tailored to specific events in the application since all application state and event information is available and known. For example, the application can cause haptic effects to execute whenever a weapon is fired, and different weapons can trigger different haptic effects. As the developer has access to the application's source code, the position of the gun in the applications internal map is also known and haptic spacialization, if available, can be accurately controlled.
This modification of an application's source code or resources, however, is not always possible. Frequently, due to proprietary and other concerns, source code or resources are not available to those desiring to add haptic support for a device to an application. If source code or resources are available for modification, the effort required to “haptically enable” a given application or set of applications may not be practical. Moreover, the source code of every application, for which haptic support is desired, needs to be modified.
There has also been some limited work on allowing haptic functionality to be triggered without requiring modification of an application's source code. Primarily, this has involved the use or analysis of the sounds generated by an application. The sounds generated are then used to trigger haptic functionality and effects. For example, the intensity of the sound generated by an application can be used to trigger haptic vibration effects whose intensity varies with that of the application's sound. The relative intensity of sounds in a stereo audio system can also be used to provide some position information about the sound sources in the audio signal. These techniques have the advantage that they do not require modification of an application or its resources in order to generate haptic effects.
These “external” techniques, however, are limited in their ability to discern specific information about events and spatial information in an application. For example, a haptic effect could execute upon the sound of a weapon being fired in an application, and the intensity of that effect could vary with the intensity of the sound. Without extensive real-time audio analysis or filtering, this same effect might also be triggered when a lion roars or a motorcycle rumbles in the application. Thus, the ability of these kind of systems to execute haptic effects and forces that are highly tailored to an application's state and events in the application is quite limited and nowhere near what can be achieved using source level integration or specific coding of audio or graphics signals and content.